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ABSTRACT 

Visually-distorted  images  can  contain  valuable  information. 
Indeed,  in  tactical  MANET  networks  where  throughput  is  ex¬ 
tremely  valuable  and  difficult  to  come  by,  guaranteeing  the 
delivery  of  every  packet  of  an  encoded  image  is  impracti¬ 
cal.  However,  designing  a  watermark  that  is  resilient  to  the 
types  of  visual  distortion  imparted  on  an  image  or  video  due 
to  channel  losses  is  a  difficult  task.  In  this  work,  we  intro¬ 
duce  a  new  watermarking  scheme  for  JPEG-compressed  im¬ 
ages  that  incorporates  ideas  from  compressed  sensing  (CS)  to 
achieve  robustness  against  certain  types  of  errors  induced  by 
noisy  communication  channels. 

This  work  uses  CS  techniques  to  embed  a  sparse  water¬ 
mark  into  L  randomly  selected  quantized  JPEG  image  co¬ 
efficients.  Sparse  reconstruction  techniques  are  then  used  to 
reconstruct  the  watermark  from  the  coefficients  that  were 
received,  including  those  that  were  incorrectly  decoded. 
Through  the  development  of  this  watermarking  scheme,  we 
would  like  to  demonstrate  and  explore  the  effect  of  the  error 
resilience  properties  of  CS  encoded  signals  on  the  image  wa¬ 
termarking  problem. We  show  through  simulation  that  even 
with  significant  visual  distortion  in  the  received  image,  the 
CS  encoded  watermark  can  be  detected  with  very  high  prob¬ 
ability. 

Index  Terms —  Watermarking,  Error  Coding,  Com¬ 
pressed  Sensing,  JPEG  Compression,  Erasure  Channel 

1.  INTRODUCTION 

Robust  digital  image  watermarking  [1],  or  the  act  of  hiding 
a  hidden  signal  within  an  encoded  image  that  is  resilient  to 
both  noise  and  tampering,  has  applications  in  both  commer¬ 
cial  and  military  systems.  In  this  work,  we  will  focus  on  mili¬ 
tary  applications  where  an  image  is  generated  at  some  remote 

Distribution  A:Public  Release.  This  work  is  sponsored  by  the 
Department  of  Defense  under  Air  Force  Contract  FA8721-05-C-0002. 
Opinions,  interpretations,  conclusions,  and  recommendations  are  those  of 
the  author  and  are  not  necessarily  endorsed  by  the  United  States 
Government. 


Scott  Pudlewski 

Lincoln  Laboratory 
Massachusetts  Institute  of  Technology 
Lexington,  MA  02420 
scott. pudlewski  @  ll.  mit.  eclu 

imaging  sensor  (i.e.,  a  persistent  surveillance  system  or  an 
unmanned  aerial  vehicle  (UAV))  and  transmitted  in  part  over 
very  lossy  tactical  MANET  links.  While  traditional  digital 
signatures  can  be  used  to  verify  that  an  image  was  received 
from  the  stated  sender,  this  may  not  go  far  enough  to  ensure 
that  the  image  was  created  by  the  sender.  In  addition,  it  may 
be  useful  to  be  able  to  verify  the  source  of  an  image  long  after 
that  images  was  transmitted,  for  instance  in  legal  proceedings 
or  to  determine  the  source  of  an  information  leak.  Based  on 
these  observations,  the  watermark  should  be  embedded  within 
the  image  data  so  that  after  any  transformation  or  distortion 
that  produces  a  useful  image,  the  watermark  is  preserved. 

In  this  work,  we  explore  the  use  of  compressed  sensing 
(CS)  [2,  3,  4]  in  robust  watermarking  of  JPEG  encoded  im¬ 
ages  [5].  While  there  are  a  number  of  alternatives  that  have 
shown  promise  in  these  scenarios  (specifically  spread  spec¬ 
trum  techniques  [6, 7]),  we  believe  that  the  combination  of  the 
error  resiliency  of  CS  encoded  signals  along  with  the  minimal 
complexity  required  for  CS  encoding  [8]  justify  its  consider¬ 
ation  as  a  watermarking  technology. 

Compressed  sensing  is  a  compression  scheme  in  which 
sparse  signal  reconstruction  techniques  are  used  to  recon¬ 
struct  a  sparse  message  signal  x  from  a  small  number  of 
noise-like  random  linear  combinations  y  =  <I>x,  where 
$  £  RMxiv,  M  <  N  is  a  noise-like  sampling  matrix. 
While  CS  as  a  compression  technique  has  a  number  of  ob¬ 
vious  applications  in  multimedia  signal  compression  [9],  in 
this  work  we  are  going  to  explore  the  impact  of  two  other 
properties  of  CS  encoded  signals.  Specifically,  we  will  look 
at  the  error  resilience  of  CS  encoded  signals,  along  with  the 
ability  to  interchange  received  CS  samples,  which  removes 
the  dependence  on  any  one  specific  sample.  In  this  work,  we 
will  show  that  these  two  properties  can  be  used  to  design  a 
watermarking  system  that  is  surprisingly  resilient  to  packet 
erasures.  The  goal  of  this  work  is  to  determine  whether  the 
error  resilience  properties  of  a  CS  encoded  watermark  justify 
the  design  of  yet  another  data  hiding  scheme.  We  believe  that, 
based  on  the  resilience  to  extremely  high  error  rates  within 
the  JEPG  encoded  image,  future  work  is  indeed  justified. 


The  remainder  of  this  paper  is  organized  as  follows.  In 
Section  2,  we  explain  the  structure  of  the  watermark  and  the 
design  of  watermarking  parameters,  while  Section  3  explains 
the  detection  scheme.  Section  4  describes  the  experimental 
setup  and  discuss  the  simulation  results.  Finally,  we  discuss 
future  work  and  draw  the  main  conclusions  in  Section  5. 

2.  WATERMARK  DESIGN 

2.1.  Encoding  the  Watermark 

To  create  the  watermark,  we  generate  a  message  vector  x  £ 
{-1, 0, 1}^  such  that  ||x||0  =  K,  N  >  K.  Let  A  £  RLxN 
be  a  full  rank  matrix  with  L  >  N.  Such  a  matrix  can  be 
created  with  full  rank  with  high  probability  [3]  using  a  seeded 
RNG  to  generate  each  entry.  The  watermark  vector  y  is  then 
generated  as  y  =  Ax. 

By  design  of  the  A  matrix,  y  is  a  real-valued  vector,  while 
the  image  coefficients  z  are  quantized  and  coded  digitally. 
Thus,  in  order  to  practically  implement  the  watermarking 
scheme,  we  need  to  quantize  y.  Using  a  pre-computed  code¬ 
book  with  Qn  quantization  steps,  we  can  find  the  quantized 
watermark  vector  y ^  =  y  +  ?),  such  that  0  <  Q(y\q •*)  <  Qn 
for  i  £  {0, 1, . . . ,  L  —  1},  where  Q(-)  is  the  quantization 
function  and  rj  represents  the  noise  introduced  by  the  quanti¬ 
zation  step. 

There  are  various  options  to  create  the  watermarked  coef¬ 
ficients  w(-).  Our  implementation  uses  integer  modulo  mask¬ 
ing: 

wi{z)=  jr-  +Q(Viq))  (1) 

for  *  £  {0, 1, . . . ,  L  —  1}.  This  is  analogous  to  k  least  signif¬ 
icant  bit  masking  when  Qn  =  2fc  for  integer  k. 

2.2.  Embedding  the  Watermark 

In  this  work  we  embed  the  watermark  vector  into  a  JPEG  en¬ 
coded  image.  However,  the  basic  watermarking  framework  is 
not  specific  to  the  compression  technique.  The  watermark  can 
be  embedded  into  any  set  of  image  transform  coefficients  (or 
even  the  original  image).  We  chose  to  use  the  JPEG  standard 
primarily  for  its  popularity. 

In  JPEG  encoding,  the  DCT  coefficients  within  each  of 
the  J  macroblocks  are  scaled  through  element-wise  integer 
division  with  quality  matrix  B(q),  where  0  <  q  <  100  is 
the  quality  level  of  the  image.  To  embed  the  watermark,  we 
modify  these  scaled  block-DCT  coefficients  of  macroblock 
intensities.  In  particular,  we  modify  the  subset  S  of  the  J 
scaled  DC  coefficients  z  of  this  block-DCT  (the  DC  coeffi¬ 
cient  of  each  macroblock  is  the  lowest  frequency  component 
of  the  DCT  of  that  macroblock). 

Let  S  C  {0, 1, . . . ,  J  —  1}  such  that  |<S|  =  L,  and  S(i) 
denotes  the  ith  element  of  S.  Set  S  of  L  image  coefficient 
indices  can  be  randomly  selected  using  the  seeded  RNG.  The 


vector  z  represents  the  quantized  DC  coefficients  with  the  wa¬ 
termark  will  be  embedded.  The  coefficients  Zs(i)  =  Wi(zg^i')) 
for  i  £  {0, 1, . . . ,  L  —  1}  are  watermarked,  while  the  coeffi¬ 
cients  z.j  =  Zj  for  j  £  S  remain  unmodified.  By  watermark¬ 
ing  a  subset  of  all  coefficients,  our  scheme  allows  for  some 
additional  security  against  falsifying.  However,  note  that  if 
L  =  J,  then  this  is  equivalent  to  modifying  all  coefficients. 

Once  the  watermarked  quantized  coefficients  z  have  been 
computed,  they  are  compressed  using  variable-length  Huff¬ 
man  encoding  and  periodically  marked  with  reset  markers 
(RST)  (to  mark  the  points  at  which  the  Huffman  decoding 
process  can  realign  in  the  case  of  corrupted  or  missing  bits). 

3.  WATERMARK  DETECTION 

In  this  work  we  assume  the  worst-case  scenario  in  that  our  wa¬ 
termark  decoder  has  no  cross-layer  information  to  determine 
which  packets  were  discarded  at  the  lower  layers.  While  some 
of  the  corrupted  macroblocks  will  have  a  zero  amplitude  and 
can  therefore  easily  be  removed,  we  can  clearly  see  in  Fig.  1 
that  many  packet  losses  will  result  in  a  “valid”  but  incorrect 
decoded  macroblock.  While  it  would  be  feasible  to  design  a 
parity  based  detection  scheme  to  find  the  errors  in  CS  encoded 
signal  (as  in  [10]),  we  show  that  even  in  this  worst-case  sce¬ 
nario  our  scheme  is  able  to  maintain  high  detection  rates  at 
high  packet  error  rates  with  moderate  sparsity  and  expansion 
levels. 

After  having  removed  the  detected  corrupt  macroblocks, 
we  have  M  watermarked  macroblocks  remaining.  Due  to  the 
use  of  RST,  the  decoder  can  determine  which  M  of  the  re¬ 
maining  blocks  originally  contained  the  embedded  watermark 
coefficients.  As  long  as  enough  blocks  are  received  such  that 
M  >  K\og(N),  the  receiver  can  correctly  reconstruct  the 
watermark. 

3.1.  Decoding  the  Watermark 

Let  U  be  the  set  of  indices  of  the  M  watermarked  block- 
DCT  coefficients  not  classified  as  corrupted  after  detection, 
and  U(i )  be  the  ith  element  of  U.  The  received  signal  y  is 
then  defined  element-wise  as 

Vi  =  Q"1  )  •  (2) 

Then  let  A  =  AHA. :)  be  the  corresponding  matrix  with  row 
indices  U  from  A.  In  other  words,  the  missing  samples  (those 
that  were  detected  as  incorrect)  are  removed  from  the  set  of 
received  samples  along  with  the  corresponding  rows  in  the 
sampling  matrix.  Since  CS  only  requires  that  enough  sam¬ 
ples  are  received  for  decoding  (i.e.,  the  samples  are  inter- 
changable),  the  watermark  will  be  able  to  be  detected  as  long 
as  enough  correct  samples  are  received  and  the  magnitude  of 
the  errors  is  bounded  and  small  [3]. 


We  then  find  x  by  solving  the  optimization  problem 


minimize  ||x||i 

X 

subject  to:  ||y  —  ^4x| 1 1  <  e, 


(3) 


which  can  be  shown  to  be  equivalent  to  finding  the  “spars¬ 
est”  vector  x  that  matches  the  received  signal  vector  y  within 
some  error  tolerance  e. 

Finally,  since  the  recovered  x  G  R  will  be  continuous¬ 
valued  while  the  original  message  only  takes  values  Xi  G 
{  — 1,  0, 1},  we  will  need  to  quantize  x.  Letting  max(x,  K) 
be  the  set  of  x-i  with  the  largest  K  absolute  values,  we  set 

x ^  =  sgn(a:i)  for  X,  G  max(x,  K)  and  =  0  otherwise. 

We  measure  the  MSE  between  the  quantized  recovered 
message  and  original  message 


||x  —  x(9)ll2 

MSE  =  (4) 

K 

From  this  definition,  it  is  clear  that  0  <  MSE  <  4.  We 
note  that  typical  values  range  from  0  <  MSE  <  2  since 
MSE  =  4  =>  x^  =  —x,  while  MSE  =  2  likely  implies 
that  the  recovery  chose  all  incorrect  indices  as  support  for  our 
estimated  message.  Thus  we  claim  to  detect  the  original  mes¬ 
sage  x  when  MSE  <  1 . 


N 

K 

P 

64 

4 

4.22  x  10"a 

64 

16 

4.45  x  10-5 

128 

4 

1.08  x  lO"1* 

128 

16 

2.67  x  10-7 

256 

4 

2.72  x  10"4 

256 

16 

1.23  x  10"y 

Table  1.  Probabilities  of  randomly  generating  detectable 
message  by  guessing  for  different  values  of  K  and  N 

given  by 

fK\  (K  —  i\  (  N-K 

WV  2  )\K-i-j 

(5) 

0  if  r  >  n.  Then  the  probabil¬ 
ity  of  generating  a  detectable  message  to  spoof  an  unauthen¬ 
ticated  watermark  is  R/T.  Probabilities  for  randomly  gener¬ 
ating  a  detectable  message  is  shown  in  Table  1 . 

This  also  shows  that  a  number  of  distinct  messages 
{xo,  Xi, . . . , xp}  can  be  designed  for  a  single  matrix  A 
to  generate  watermarks  for  P  different  authentic  sources 
without  being  detected  as  a  different  authentic  message. 


K  i-\K/i\ 

r  =  2 K~l~j 

i=\K/2]  j—0 


where  by  convention 


3.2.  Selecting  Watermark  Parameters 

Since  this  scheme  requires  the  watermark  to  be  restricted  to  be 
a  I\  -sparse  signal,  we  need  to  verify  that  that  restriction  does 
not  make  the  resulting  watermark  too  “easy”  for  an  adversary 
to  guess  using  brute-force.  We  wish  to  design  our  watermark 
such  that  a  third  party  who  knows  the  watermarking  scheme 
but  does  not  have  full  information  about  the  shared  secrets  (A 
or  x)  will  not  be  able  to  generate  a  “correct”  but  unauthorized 
watermark.  In  the  case  where  the  third  party  only  has  knowl¬ 
edge  of  the  message,  then  the  matrix  A  must  be  guessed. 
However,  since  guessing  each  element  of  the  sampling  ma¬ 
trix  A  in  infeasible1,  we  only  consider  the  more  tractable  case 
in  which  the  third  party  has  knowledge  of  the  matrix  A  but 
not  of  the  message  x. 

The  total  number  of  messages  of  length  N  and  sparsity  K 

is  T  =  2k  ■  Assuming  that  every  message  has  an  equal 

chance  of  being  generated  by  the  third  party,  the  probability  of 
guessing  the  exact  message  is  the  reciprocal  of  this  quantity. 
However,  since  we  do  not  require  perfect  recovery  for  detec¬ 
tion  of  the  message,  we  examine  the  probability  of  guessing 
a  detectable  message.  The  number  of  detectable  messages  is 

1  While  our  preliminary  studies  have  shown  that  some  information  can  be 
gained  if  an  attacker  can  correctly  guess  80%  of  the  sampling  matrix  elements 
within  e,  for  even  our  smallest  case,  the  sampling  matrix  A  is  a  real  valued 
64  x  256  matrix.  This  would  require  an  attacker  to  correctly  guess  at  least 
13,108  real  valued  elements,  which  is  clearly  not  practical. 


4.  SIMULATIONS  AND  RESULTS 
4.1.  Experiments 

We  use  the  classic  512  x  512  grayscale  Lenna  test  image  [11] 
for  our  simulations,  and  gradient  projection  for  sparse  recon¬ 
struction  (GPSR)  [12]  to  solve  the  convex  optimization  prob¬ 
lem.  We  chose  a  packet  size  of  128  bytes  and  inserted  a  RST 
marker  every  64  macroblocks  which  corresponds  to  one  at 
the  end  of  each  horizontal  row.  We  assume  the  JPEG  header 
is  transmitted  intact. 

Figure  1  shows  a  typical  watermarked  image  and  a  re¬ 
ceived  image  at  PER  =  0.18.  For  this  work,  we  assume  a 
Gaussian  packet  erasure  channel.  We  see  the  structure  of  the 
errors  from  the  lost  packets  in  1(b).  Note  that  even  in  the 
presence  of  relatively  high  PER,  there  is  some  visually  signif¬ 
icant  information  present  in  the  transmitted  image  that  may 
be  worth  verifying. 

Figure  2  shows  the  watermark  recovery  step  performed 
on  a  typical  instance  of  a  received  image.  The  recovered  x  is 
continuous- valued  but  sparse  with  the  correct  support,  so  the 
quantization  to  x9  still  matches  the  original  message  exactly 
and  is  detectable. 

In  our  simulations,  we  examined  the  effects  of  several  pa¬ 
rameters  of  the  detection  rate  of  the  watermark  across  varying 
PER.  These  parameters  included  sparsity  level  N/K,  expan¬ 
sion  level  L/N,  and  number  of  quantization  steps  Qn.  The 


Fig.  1.  (a)  shows  the  watermarked  image  with  no  transmission 
errors;  (b)  shows  the  received  image  at  PER  =  0.18. 
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Fig.  2.  Recovery  step 


Detection  of  Watermark  vs  Packet  Error  Rate  for  varying  Expansion  Levels 


Fig.  4.  Detection  rate  across  packet  error  rates  at  varying  ex¬ 
pansion  levels  L/N 


Detection  of  Watermark  vs  Packet  Error  Rate  for  varying  Quantization  Level 


detection  rate  for  each  set  of  parameters  was  calculated  as  the 
ratio  between  the  number  of  times  the  true  watermark  was 
detected  and  the  total  number  of  trials.  Each  setting  was  run 
with  100  Monte  Carlo  trials. 

Figure  4  shows  that  the  detection  rates  across  all  packet 
error  rates  can  be  increased  by  increasing  the  expansion  level 
L/N.  Figure  3  similarly  shows  that  detection  rates  across 
packet  error  rates  can  be  increased  by  increasing  the  sparsity 
N/K  of  the  message  x.  Comparing  this  result  to  the  analysis 
from  Table  1 ,  we  see  that  there  is  a  tradeoff  between  security 
and  error  protecting  performance  in  designing  the  sparsity  of 
the  message. 


Fig.  5.  Detection  rate  across  packet  error  rates  at  varying 
quantization  levels  Qn 

Figure  5  demonstrates  increasing  the  quantization  levels 
used  to  embed  the  watermark  can  also  increase  performance. 
However,  there  seems  to  be  a  point  at  which  this  increase  no 
longer  results  in  significant  increase  in  detection  rate.  This 
could  be  due  to  the  fact  that  increasing  the  quantization  levels 
increases  the  magnitude  of  the  gross  errors. 

5.  CONCLUSION 


Detection  of  Watermark  vs  Packet  Error  Rate  for  varying  Sparsity  Levels 


Fig.  3.  Detection  rate  across  packet  error  rates  at  varying  spar¬ 
sity  levels  N/ K 


In  this  work  we  introduced  the  application  of  CS  to  robust 
image  watermarking.  This  was  done  by  introducing  a  wa¬ 
termarking  scheme  for  JPEG  encoded  images  and  showing 
that  it  was  effective  in  noisy  communication  channels.  Our 
scheme  takes  advantage  of  sparse  recovery  ideas  borrowed 
from  compressed  sensing  to  achieve  performance  in  very 
noisy  conditions.  We  demonstrated  this  performance  under 
certain  parameters  and  analyzed  the  security  of  the  authen¬ 
tication  provided  by  the  scheme.  Future  work  includes  ex¬ 
tending  our  scheme  to  video  watermarking.  We  believe  our 
scheme  is  particularly  well-suited  for  streaming  video,  since 
our  protocol  can  handle  discarded  packets  and  does  not  rely 
on  knowledge  of  the  original  image.  In  addition,  alternate 
embedding  techniques  and  types  of  attacks  could  be  studied. 
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